1. Field of the Invention
The present invention is related to data processing systems using neural network models in computer architecture systems.
2. Description of the Related Art
Computer architecture systems have been developed recently which use a parallel processing scheme known as neural networks.
A neural network to be used in a data processing system is constructed in a multi-layer state by placing in parallel a neural cell such as the neural cell model 10 shown in FIG. 1 ("neuron", hereinafter). The neuron in each layer is connected by synapses to all the neurons in an adjacent layer, and the neuron inputs and outputs data. Referring to neuron 10 FIG. 1, weights W1, W2, W3, . . . , Wn are multiplied to data I1, I2, I3, . . . , In which are externally input. Data O, which corresponds to the comparison between the sum of the multiplication and threshold .THETA., is output from the neuron 10.
Various methods are possible for the comparison operation of the neuron 10. When the normalized function 1 [f] is adopted, output data O is expressed as shown in formula (1). EQU O=1[.SIGMA.Wn.multidot.In-.THETA.] (1)
When .SIGMA.=Wn.multidot.In is more than or equal to threshold .THETA., the neuron ignites and the output data O is "1"; when Wn.multidot.In is less than threshold .THETA., the neuron does not ignite and the output data O is "0".
Conventional neural networks form a neural layer by placing such neurons in parallel and by connecting the neural layers in series. A neural layer is comprised of, for example, three layers of an input layer, middle layer and output layer as Perceptrons: as suggested by Rosenblatt, the neuron in each layer combines with all neurons in the adjacent layers by synapses.
In such a data processing system, the operation of adapting the weight of the synapse of each neuron is called "learning". Since it is important when performing the learning operation to guarantee its realization and to perform it efficiently, problems associated with these factors may rise. For example, in back-propagation methods of learning which have attracted attention in previous years, problems exist in obtaining a local minimum and a small convergence time. These problems are magnified because the tendency is strong in neural networks to include a large number of middle layers.